1. Field of the Invention
The present invention relates to image processing methods and systems for correcting the image quality of a moving image, and programs for carrying out the methods.
2. Description of the Related Art
Correcting original digital images for image quality has been widely performed so that the digital images look attractive. Typical examples are white balance correction, exposure correction, color saturation correction, contrast correction, sharpness correction, color correction, etc. As a method of correcting a still image, it is a well known technique to extract a feature quantity from the still image, calculate a correction value based on the extracted feature quantity, and correct the original image with the correction value. The feature quantity to be extracted from a still image can employ an average luminance value for the original image, an average luminance value in a dark region within the original image, an average luminance value for a main photographed body (person's face, etc.) in the original image, an average saturation value in a high saturation region within the original image, an average gradation value for red, green, and blue (RGB) components in a light region within the original image, a hue histogram within the original image, etc., depending on the content of correction processes, such as white balance correction, color saturation correction, etc.
The aforementioned method for correcting the image quality of a still image can be applied to the correction of the image quality of a moving image. That is, a moving image can be corrected by considering each of the picture frames of the moving image as a still image, then extracting a feature quantity from each picture frame, then calculating a correction value for each frame image based on the extracted feature quantity, and correcting each picture frame with the correction value.
However, the picture frames of a moving image vary in a temporal direction and the feature quantities also vary, so if the still-image correcting method is performed on each of the picture frames, flickering of a corrected moving image comprising the corrected picture frames will occur. To avoid the flickering of a corrected moving image, a method for smoothing variation of correction values for the picture frames of a moving image is disclosed in U.S. Patent Application Publication No. 20020145678. More specifically, there has been proposed a method for inhibiting the flickering of a corrected moving image by limiting the difference between a correction value for the present picture frame and a correction value for the previous picture frame to less than a predetermined threshold value. There is also a method for using as a correction value for the present picture frame a value obtained by taking the average (moving average) of (1) a correction value for the present picture frame and (2) a correction value for the picture frame before the present picture frame and/or a correction value for the picture frame after the present picture frame.
For moving images, there are different scenes such as interior scenes and exterior scenes, and the respective picture frames of these different scenes have no connection with each other. Therefore, if variation of correction values for picture frames is inhibited in order to prevent the flickering of a corrected moving image, an object correction value for a picture frame that becomes a cut point (the first and last picture frames of one scene) representing a switch from one scene to another will be influenced by a correction value for a picture frame of a different scene. For example, the last picture frame of scene-1 and the first picture frame of the following scene-2 are picture frames adjacent in the temporal direction, but they have no connection with each other. Therefore, if an object correction value is obtained, for example, by taking the moving average of correction values for picture frames to avoid the flickering of a corrected moving image, an object correction value for the last picture frame of the scene-1 is obtained by taking the average of this picture frame and the two picture frames before and after that frame (including the first picture frame of the scene-2). Thus, the object correction value for the last picture frame of the scene-1 is unsuitable. To solve this problem, the aforementioned U.S. Patent Application Publication No. 20020145678 discloses a method that does not use a correction value for a picture frame of a scene differing from an picture frame of interest, when detecting a cut point that represents a switch from one scene to another in a moving image, and smoothing variation of correction values for picture frames to calculate an object correction value for a picture frame.
However, a method for smoothing variation of correction values, such as that disclosed in the aforementioned U.S. Patent Application Publication No. 20020145678, inhibits fluctuations only with respect to correction values, and can eliminate the flickering of correction values when a fluctuation in the temporal direction of a picture frame is slight, but cannot eliminate the flickering of a corrected moving image.
In digital still cameras (DSCs), corrections are sometimes made in obtaining a moving image by photography. When these corrections are inappropriate, discontinuous corrections will have an adverse influence on the image quality of the moving image.
On the other hand, in a moving image, there are cases where picture frames vary rapidly within the same scene. As shown in FIG. 19, for instance, three picture frames A, B, and C, obtained by photographing a person moving at high speeds in a close-up, are picture frames within the same scene, but the content of the image varies considerably between them. The method in the aforementioned U.S. Patent Application Publication No. 20020145678 can calculate an appropriate correction value for a picture frame that becomes a cut point. However, if in calculating an object correction value for the picture frame B shown in FIG. 19, it is decided that the picture frame B is a cut point, and a correction value calculated from the picture frame B is used as an object correction value for the picture frame B, the flickering of a corrected moving image in the same scene will occur because the object correction values between the picture frame B and the picture frames A, C are not smoothed. On the other hand, if it is decided that the picture frame B is not a cut point, and the average value of the correction values for the picture frames A, B, and C is used as an object correction value for the picture frame B, the image quality is not good because differences among the picture frame B and the picture frames A, C are not reflected.